92 research outputs found

    Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Media

    Full text link
    We present an overview of the third edition of the CheckThat! Lab at CLEF 2020. The lab featured five tasks in two different languages: English and Arabic. The first four tasks compose the full pipeline of claim verification in social media: Task 1 on check-worthiness estimation, Task 2 on retrieving previously fact-checked claims, Task 3 on evidence retrieval, and Task 4 on claim verification. The lab is completed with Task 5 on check-worthiness estimation in political debates and speeches. A total of 67 teams registered to participate in the lab (up from 47 at CLEF 2019), and 23 of them actually submitted runs (compared to 14 at CLEF 2019). Most teams used deep neural networks based on BERT, LSTMs, or CNNs, and achieved sizable improvements over the baselines on all tasks. Here we describe the tasks setup, the evaluation results, and a summary of the approaches used by the participants, and we discuss some lessons learned. Last but not least, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research in the important tasks of check-worthiness estimation and automatic claim verification.Comment: Check-Worthiness Estimation, Fact-Checking, Veracity, Evidence-based Verification, Detecting Previously Fact-Checked Claims, Social Media Verification, Computational Journalism, COVID-1

    Overview of the CLEF-2022 CheckThat! Lab Task 1 on Identifying Relevant Claims in Tweets

    Get PDF
    We present an overview of CheckThat! lab 2022 Task 1, part of the 2022 Conference and Labs of the Evaluation Forum (CLEF). Task 1 asked to predict which posts in a Twitter stream are worth fact-checking, focusing on COVID-19 and politics in six languages: Arabic, Bulgarian, Dutch, English, Spanish, and Turkish. A total of 19 teams participated and most submissions managed to achieve sizable improvements over the baselines using Transformer-based models such as BERT and GPT-3. Across the four subtasks, approaches that targetted multiple languages (be it individually or in conjunction, in general obtained the best performance. We describe the dataset and the task setup, including the evaluation settings, and we give a brief overview of the participating systems. As usual in the CheckThat! lab, we release to the research community all datasets from the lab as well as the evaluation scripts, which should enable further research on finding relevant tweets that can help different stakeholders such as fact-checkers, journalists, and policymakers

    Fighting the COVID-19 Infodemic:Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society

    Get PDF
    With the emergence of the COVID-19 pandemic, the political and the medical aspects of disinformation merged as the problem got elevated to a whole new level to become the first global infodemic. Fighting this infodemic has been declared one of the most important focus areas of the World Health Organization, with dangers ranging from promoting fake cures, rumors, and conspiracy theories to spreading xenophobia and panic. Addressing the issue requires solving a number of challenging problems such as identifying messages containing claims, determining their check-worthiness and factuality, and their potential to do harm as well as the nature of that harm, to mention just a few. To address this gap, we release a large dataset of 16K manually annotated tweets for fine-grained disinformation analysis that (i) focuses on COVID-19, (ii) combines the perspectives and the interests of journalists, fact-checkers, social media platforms, policy makers, and society, and (iii) covers Arabic, Bulgarian, Dutch, and English. Finally, we show strong evaluation results using pretrained Transformers, thus confirming the practical utility of the dataset in monolingual vs. multilingual, and single task vs. multitask settings

    ACCESS & LRG-BEASTS: a precise new optical transmission spectrum of the ultrahot Jupiter WASP-103b

    Full text link
    We present a new ground-based optical transmission spectrum of the ultrahot Jupiter WASP-103b (Teq=2484T_{eq} = 2484K). Our transmission spectrum is the result of combining five new transits from the ACCESS survey and two new transits from the LRG-BEASTS survey with a reanalysis of three archival Gemini/GMOS transits and one VLT/FORS2 transit. Our combined 11-transit transmission spectrum covers a wavelength range of 3900--9450A with a median uncertainty in the transit depth of 148 parts-per-million, which is less than one atmospheric scale height of the planet. In our retrieval analysis of WASP-103b's combined optical and infrared transmission spectrum, we find strong evidence for unocculted bright regions (4.3σ4.3\sigma) and weak evidence for H2_2O (1.9σ1.9\sigma), HCN (1.7σ1.7\sigma), and TiO (2.1σ2.1\sigma), which could be responsible for WASP-103b's observed temperature inversion. Our optical transmission spectrum shows significant structure that is in excellent agreement with the extensively studied ultrahot Jupiter WASP-121b, for which the presence of VO has been inferred. For WASP-103b, we find that VO can only provide a reasonable fit to the data if its abundance is implausibly high and we do not account for stellar activity. Our results highlight the precision that can be achieved by ground-based observations and the impacts that stellar activity from F-type stars can have on the interpretation of exoplanet transmission spectra.Comment: 33 pages, 17 figures, 7 tables. Accepted for publication in A
    • …
    corecore